A conventional noise elimination apparatus attenuates the interframe difference of a video signal at a ratio determined in accordance with conditions, and removes noise contained in the difference values. For example, in Patent Document 1, a motion deciding unit detects motion according to values based on the difference values to determine a ratio K (0<K<1), and obtains pixel values by adding previous frame pixel values multiplied by the ratio K and input pixel values multiplied by the ratio (1−K), thereby obtaining pixel values that attenuate noise contained in the interframe differences. Such a processing section is referred to as a noise elimination filter.
For example, as shown in the Patent Document 1, the conventional noise elimination apparatus applies the input pixel values and the previous frame pixel values to noise elimination calculation processing based on a ratio of (1−K):K. In other words, assume that the frame number proceeds with the time elapsed, and that the input pixel value (input image) of a given frame number t in the noise elimination processing is ft, and the pixel value after the noise elimination is Ft, then Ft is given by the following expression (1). Here, the frame number t is assumed to be an integer, and Ft-1 designates a pixel value (reference image) of the previous frame after the noise elimination.Ft=(1−K)·ft+K·Ft-1  (1)
Although not described in the Patent Document 1, the video signal is generally handled in terms of integers. Accordingly, assuming that int(x) is an integerization function for integerizing a real number x, the integerized pixel value Ft after the noise elimination is given by the following expression (2).Ft=int((1−K)·ft+K·Ft-1)  (2)
In this case, since both Ft-1 and Ft are integers, the foregoing expression (2) can be rewritten as the following expression (3) which is expressed as the sum of the integerized pixel value Ft-1 after the noise elimination of the previous frame and the value (referred to as “output difference value”) obtained by multiplying the difference value (ft−Ft-1) between the input pixel value and the previous frame pixel value by the ratio (1−K).Ft=Ft-1+int((1−K)·(ft−Ft-1))  (3)
FIG. 1 and FIG. 2 are diagrams each showing an example of nonlinear characteristics of an integerization function applied portion for the difference value (ft−Ft-1) between the input pixel value ft and the previous frame pixel value Ft-1 in the foregoing expression (3). In FIG. 1 and FIG. 2, the horizontal axis represents the difference value (ft−Ft-1) between the input pixel value and the previous frame pixel value. Here, the noise elimination is implemented by switching, as to the difference value (ft−Ft-1), the ratio (1−K) between an interval near zero where even minute noise is perceivable and intervals on both sides of the interval near zero, where the minute noise has little effect and is hard to be perceived. Here, the following description will be made on the assumption that the noise near zero is eliminated.
For example, in FIG. 1 and FIG. 2, the interval in which the absolute value of the difference value (ft−Ft-1) is equal to or less than four is defined as the interval near zero. In this interval, (1−K) is set at 1/4, and in the intervals in which the absolute value of the difference value is greater than four, (1−K) is set at one. In this case, the calculation results of the real number (1−K)·(ft−Ft-1) are shown in broken lines.
The integerization function int(x) includes integerization based on the rounding down processing and rounding up processing, and FIG. 1 shows a case of integerization based on the rounding down processing, where as FIG. 2 shows a case of integerization based on the rounding up processing. For the real number calculation results shown by broken lines, the integer values int((1−K)·(ft−Ft-1)) are represented by solid lines. In actuality, since the difference value (ft−Ft-1) is an integer, they have values only on lattice points whose values on the horizontal axis are integers. It is assumed that open circles do not include the endpoints, whereas solid circles include the endpoints. As shown in FIG. 1 and FIG. 2, depending on whether the integerization is carried out according to the rounding down processing or to the rounding up processing, different noise elimination characteristics are obtained.
First, removal of minute noise will be described in the case where the integerization is carried out by the rounding down processing and rounding up processing. FIG. 3 is a diagram showing an example of percentages of the difference values (ft−ft-1) between corresponding input pixel values in consecutive frames to the component pixel numbers of respective frames when a fixed camera takes a photograph of an object at rest. Generally, some difference takes place in the input pixels because of addition of input system noise to the input pixels. In the example of FIG. 3, it is shown that pixels with the difference of magnitude 2 are 20%, pixels with the difference of magnitude 1 are 30%, and pixels without the difference of magnitude 0 are remaining 50%, and that the percentages are nearly constant.
With respect to FIG. 3, the minute noise removal characteristics will be described when the integerization is carried out using the rounding down processing shown in FIG. 1 as the integerization function.
When the difference values (ft−ft-1) between the corresponding input pixel values of consecutive frames take place with the percentages as shown in FIG. 3, the percentage of the difference values (Ft−Ft-1) between pixel values of the consecutive frames after the noise elimination will become as shown in FIG. 4 by performing integerization based on the rounding down processing. As shown in FIG. 1, in the case of carrying out integerization by the rounding down processing when 1−K=1/4, since the pixels with the difference values (ft−Ft-1) of magnitude 1 or 2 become zero through the calculation processing of int((1−K)·(ft−Ft-1)), the foregoing expression (3) yields Ft=Ft-1. Accordingly, as shown in FIG. 4, the difference values (Ft−Ft-1) between the corresponding pixel values of the consecutive frames after the noise elimination become all zero, thereby eliminating noise.
Next, with respect to FIG. 3, the minute noise removal characteristics will be described when the integerization is carried out using the rounding up processing shown in FIG. 2 as the integerization function.
When the difference values (ft−ft-1) between the corresponding input pixel values of consecutive frames take place with the percentages as shown in FIG. 3, the percentages of the difference values (Ft−Ft-1) between pixel values of the consecutive frames after the noise elimination will become as shown in FIG. 5 by performing integerization based on the rounding up processing. As shown in FIG. 2, in the case of carrying out integerization by the rounding up processing when 1−K=1/4, since the pixels with the difference values (ft−Ft-1) of magnitude 1 or 2 become 1 through the calculation processing of int((1−K)·(ft−Ft-1)), the foregoing expression (3) yields Ft=Ft-1+1. Accordingly, as shown in FIG. 2, the difference values (Ft−Ft-1) with magnitude 2 between the corresponding pixel values of the consecutive frames after the noise elimination become all 1, thereby leaving some noise without eliminating completely, although the noise of the frame is reduced as a whole.
The foregoing is the description about the effect on the removal of minute noise due to the difference between the rounding down processing and the rounding up processing in the integerization.
Next, the effect on the removal of minute noise due to the difference between the rounding down processing and the rounding up processing in the integerization will be described in the case where a screen gradually makes a transition from a certain still image to another different still image because of transition such as a dissolve in a conventional noise elimination apparatus. It is assumed here that (1−K) in the integerization function in the foregoing expression (3) is 1/4 when the absolute value of the difference value (ft−Ft-1) is equal to or less than four, and is one when it exceeds four as in FIG. 1 and FIG. 2.
FIG. 6 is a graph showing an example of the input pixel values ft of the image data at any given frame number t and of the pixel values Ft having undergone the noise elimination. In the graph at the top of FIG. 6, crosses denote the input pixel values ft whose magnitude varies one by one from zero to eight over eight frames from t=5 to t=12, thereby making a transition to another still image. This example will be described.
First, a case that applies the rounding down processing shown in FIG. 1 as the integerization function will be described. The output pixel values Ft corresponding to the input pixel values ft make transition as denoted by squares in the graph at the top of FIG. 6. The difference values (ft−Ft-1) in the course of the transition are denoted by squares in the graph at the bottom of FIG. 6.
In the graph at the bottom of FIG. 6, int((1−K)·(ft−Ft-1)) in the foregoing expression (3), which is an increment from the output pixel value Ft-1 of the previous frame, is obtained by multiplying the differences denoted by the squares by (1−K), that is, by 1/4, followed by integerization by the rounding down processing. Accordingly, in the range from t=8 to t=12 in which the difference values (ft−Ft-1) take four, the increment becomes one. In this case, the output pixel values Ft in the graph at the top of FIG. 6 increase one by one up to five. In the ranges of t<8 and t>12 in which the difference values (ft−Ft-1) are less than four, the increment becomes zero so that the output pixel values Ft in the graph at the top of FIG. 6 do not increase.
At t=12, the transition of the input pixel values ft ends. The difference values with magnitude 4 up to t=12 become magnitude 3 at t=13 at which they are changed to reduce. From that point on, although the difference values are considered to be minute noise and are eliminated, the output pixel values Ft never exceed five and end the transition with being maintained at five, which is perceived as afterimages.
Next, a case that applies the rounding up processing shown in FIG. 2 as the integerization function will be described. The output pixel values Ft corresponding to the input pixel values ft make transition as denoted by circles in the graph at the top of FIG. 6. The difference values (ft−Ft-1) in the course of the transition are denoted by circles in the graph at the bottom of FIG. 6.
In the graph at the bottom of FIG. 6, int((1−K)·(ft−Ft-1)) in the foregoing expression (3), which is an increment from the output pixel value Ft-1 of the previous frame, is obtained by multiplying the differences denoted by the circles by (1−K), that is, by 1/4, followed by integerization by the rounding up processing. Accordingly, in the range from t=5 to t=12 in which the difference values (ft−Ft-1) take one, the increment becomes one. In this case, the output pixel values Ft in the graph at the top of FIG. 6 increase one by one up to eight with taking the same values as the input pixel values ft denoted by the crosses. Thus, any afterimage is not perceived.
Furthermore, a case that applies the integerization function when the difference between the input pixel values and the output pixel values of the previous frame is equal to or greater than two will be described. In the case of applying the rounding down processing shown in FIG. 1 as the integerization function, afterimages will be perceived because the output pixel values Ft cannot make a transition up to the same value as the input pixel value at the end of the transition as in the case where the difference is one.
Likewise, in the case where the rounding up processing shown in FIG. 2 is applied as the integerization function at the time when the difference between the input pixel values is equal to or greater than two, although the transition begins simultaneously with the start of the transition of the input pixel value, an increment of the difference is reduced because the difference is multiplied by (1−K). Thus, although they do not take the same values as in the transition process when the difference is one, since the output pixel values Ft can make transition up to the same value as the end value of the transition of the input pixel values, no afterimage is perceived.
Here, the description is made by way of example of the dissolve that makes a linear transition with the difference of constant magnitude. However, a similar phenomenon can be confirmed as to the minute noise even in the nonlinear transition with the difference of variable magnitude.
As described above, when applying as the integerization function the rounding down processing to the calculation processing of the noise elimination, although the conventional noise elimination apparatus can eliminate minute noise, afterimages are sometimes perceived. On the other hand, when applying the rounding up processing as the integerization function, although it can prevent afterimages from being perceived, it cannot sometimes eliminate the minute noise.
Patent Document 1: Japanese Patent Laid-Open No. 6-225178/1994 (Paragraph 0013, FIGS. 1 and 6).
With the foregoing configuration, the conventional noise elimination apparatus has a problem of being unable to achieve the removal of minute noise and the elimination of afterimages at the same time.
The present invention is implemented to solve the foregoing problem. Therefore it is an object of the present invention to provide a noise elimination apparatus and noise elimination method capable of removing the minute noise and eliminating afterimages at the same time.